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^ Abstract 
> 

QQ We explore methods to use thermostatically controlled loads (TCLs), such as wa- 

^ ter heaters and air conditioners, to provide ancillary services by assisting in bal- 

^ ancing generation and load. We show that by adding simple imbedded instructions 

^ and a small amount of memory to temperature controllers of TCLs, it is possible 

> to design open-loop control algorithms capable of creating short-term pulses of 

^ demand response without unwanted power oscillations associated with temporary 

synchronization of the TCL dynamics. By moving a small amount of intelligence 
to each of the end point TCL devices, we are able to leverage our knowledge of the 
time dynamics of TCLs to shape the demand response pulses for different power 



* Corresponding author 

Email addresses: ns init syn@ lanl . gov (Nikolai A. Sinitsyn), 
soumyakgumich . edu (Soumya Kundu), backhausSlanl . gov (Scott Backhaus) 



Preprint submitted to Energy Conversion and Management 



November 2, 2012 



system applications. A significant benefit of our open-loop method is the reduc- 
tion from two-way to one-way broadcast communication which also eliminates 
many basic consumer privacy issues. In this work, we focus on developing the 
algorithms to generate a set of fundamental pulse shapes that can subsequently be 
used to create demand response with arbitrary profiles. Demand response control 
methods, such as the one developed here, open the door to fast, nonperturbative 
control of large aggregations of TCLs. 

Keywords: Thermostatically-controUed-loads, demand side control, ancillary 
services, generation-load balancing, hysteresis-based control 

1. Introduction 

Conventional thermal generators, which are constrained by physical ramp rates, 
have difficulty in manoeuvering to compensate for variability in loads or other 
generation, e.g. variability due to time-intermittent generation such as wind gen- 
eration. On the other hand, electrical loads, e.g. thermostatically-controUed-loads 
(TCLs), may be a better alternative to provide the required generation-balancing 
ancillary services [[Bll[lll[5l[ai7l[l|9l[l0l[IIl[l2l[Il[Il[l5l because it is 
feasible for these loads to compensate for power imbalances much more quickly 
than thermal generators. TCLs such as air conditioners (AC) and water heaters 
account for about 50% of electricity consumption in the United States [ |T6| im- 
plying that the potential resource provided by TCLs is quite large. A residential 
AC unit consumes 0.5-7 kW, and when aggregated over a large city with a mil- 
lion houses, the total controllable resource approaches ~3 GW. The typical TCL 
duty cycle will limit the amount of resource available at any one time, but even a 
relatively low duty cycle of 10% still yields a controllable resource of ~300MW. 
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Because these TCL loads are on/off loads and not subject to the ramping limits of 
conventional thermal generation, they can be fully deployed on the time scale of 
the latency of the communication system used to access them, e.g. ~0.3 minutes 
(31 [171 [181 [191 HOI. Such short time scales for full deployment are not achievable 
with spinning or non-spinning reserves and, therefore, TCLs have considerable 
potential to provide both system reserves and generation-load balancing ancillary 
services [I2ni22]| . 

Past research on controlling TCLs has been either on direct interruption for 
long (several hours) duration [[TSl l23l l24l l25ll or on shifting of the TCLs tempera- 
ture set point lIlTl l26l l27l l28l . The infrequent application of the former technique 
is useful for critical peak shaving, but frequent application would quickly lead to 
customer fatigue. An approach based on small shifts of the TCL set point appears 
more attractive. In this case, subtle (0.1-0.5°C) changes in the thermostat set point 
temperature can create significant changes in aggregate power consumption. Such 
small set point shifts would be almost unnoticed by customers and could be ap- 
plied frequently as long there is not a long-term drift in one direction. However, 
this method is not without its difficulties. Extensive analytical and numerical mod- 
eling studies have shown that suddenly imposed changes to the operation cycles of 
a large population of TCL loads leads to temporary synchronization of otherwise 
uncorrected autonomous devices resulting in potentially detrimental power oscil- 
lations flm |26l [271 |28l|29l. Efforts to resolve this problem by feedback control 
techniques encounter additional problems, e.g. the need for two-way communi- 
cation to provide full state information for each TCL and subsequent individual 
control of all TCLs. This approach, at least, requires development of new software 
that would process large amount of data on-fly while the state estimation must be 
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done in a noisy environment created by all the other loads on the system [|30l . 
Perhaps more important is that two-way communication leads to privacy security 
issues that have already raised a popular protest OTl l32l [33l l34l . It seems that it 
can be implemented only on the voluntary basis, possibly with only a small frac- 
tion of TCL owners willing to lower their security level in return for rebate. For 
example, if only 5% from 1 million TCL owners in the area allow for the two-way 
communication with their utility company, this would lead to insignificant avail- 
able power for control. However, the information collected from this 5% of the 
population should be sufficient to estimate some important aggregated parameters 
of the full population, such as the fraction of the currently working TCLs, the to- 
tal power consumed by these TCLs, distribution of the temperature band widths 
of their operation, fraction of TCLs in the ON and OFF modes e.t.c. Extraction 
of this information, arriving to the utility company from only 5% of all TCLs, 
does not require a complex software and can be achieved "on-fly". Therefore, we 
believe that the technology that would allow for reliable one-way communication 
control of the remaining "95% of TCLs", and that would be based on the knowl- 
edge of only basic aggregate parameters of the full population of TCLs, should be 
attractive for large scale commercial application. 

In this article, we propose an open loop approach which eliminates the prob- 
lem of unwanted synchronization of TCLs without a need for two-way commu- 
nication opening a path for implementing control functions with a heterogeneous 
population of TCLs using existing technologies. We show that by adding a small 
amount of intelligence (in the form of a few new elementary instructions) to the 
endpoint TCLs, we are able to implement simple open-loop controls which do not 
result in subsequent TCL load synchronization. Such strategies to control large 
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heterogeneous TCL populations, which we will call the safe protocols (SPs), could 
be quickly implemented to compensate for a generation-load mismatch without 
the fear of potentially dangerous subsequent TCL behavior. In [35], one such a 
SP for shifting temperature set point of the population of loads, without initiating 
any unwanted power oscillations, was proposed. The mechanism was shown to 
provide effective control on a relatively slow time scale (e.g. ~30 minutes). Here 
we extend the idea and show that embedded instructions allow to "safely" gener- 
ate a class of power pulses of several basic shapes. For example, we will show 
that it is possible to generate a sharp (e.g. 3 minutes long) power pulse with subse- 
quent slow return of all TCLs to the initial customer set values of the temperature 
band. At the other extreme, we demonstrate another safe protocol that generates 
time-extended pulses (~0.5-l hour) with changes in the TCL controlled tempera- 
tures of below 1°C. Such pulses are useful in spinning reserve applications. In this 
work, we focus on the creation of these fundamental shapes and we postpone the 
discussion of optimal pulse reconstruction and state estimation for future studies. 

Our article is arranged as follows: In Section|2} we will outline the major prob- 
lems that appear with straightforward (not safe) strategies to control the power of 
TCLs; in Section [3| a demonstration of a SP is presented. In Section]?} we ex- 
tend the idea and present a second protocol that achieves a shift of temperature set 
points of TCLs; Section |5] presents a SP that creates a sharp power pulse of short 
duration; while Section [6] briefly reviews possible further strategies to combine 
different SPs to achieve a desired power response. We summarize our results in 
Section |7l 
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2. Problem Description 

Before we proceed with introduction to SPs, we outline the basic problems en- 
countered when more straightforward "unsafe" approaches are applied to control 
the power consumption of heterogeneous TCL populations. 




(a) Dynamics of temperature (top) and con- (b) TCL probability distribution function (PDF) 
sumed power (bottom) of a thermostatic load 

Figure 1 : (left) Individual dynamics and (right) aggregate steady state probability distribution of 
TCLs over temperature in the OFF (red) and ON (blue) modes. Small nonzero density of TCLs 
beyond the deadband limits in (b) is due to the combined effect of temperature fluctuations and a 
finite response time to temperature measurements. 



When not subject to major thermal disturbances, a TCL's power consumption 
is cyclic in nature. If we consider a TCL providing cooling, the controlled tem- 
perature 9 drops when the TCL draws power (in the ON state) and increases when 
the TCL is not drawing power (in the OFF state). Evolution of temperature in a 
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single house is generally modeled by a differential equation [|26ll : 

-^{0 -Oa^b + PR), ON State 



(1) 



'^{0-9amb), OFF State 



where Oamb is the ambient temperature, C is the thermal capacitance, R is the 



Current OFF state PDF 

Current ON state PDF 

Previous OFF state PDF 

Previous ON state PDF 

Direction of migration of OFF TCLs 
Direction of migration of ON TCLs 
Natural ON-to-OFF switching at lower boundary 
^ Natural OFF-to-ON switching at upper boundary 
A Forced ON-to-OFF switching 
y Forced OFF-to-ON switching 

Figure 2: A list of different signs used in the following figures. 

thermal resistance, and P is the cooling power of the TCL when in the ON state. 
P is proportional to the power drown from the Power Grid with a coefficient that 
describes the efficiently of the TCL. We assume the regime of TCL operation with 
a duty cycle less than 100%. Such a TCL switches its state from OFF to ON when 
its temperature increases to 0^ and from OFF to ON when temperature drops to 
0_ . Such dynamics and control push the TCL into a hysteresis around the set point 
temperature Og = (9- +9+) /2 with a dead band of A = (9+ — Solution 
of Eqs. ([T]) with such boundary conditions is shown in Fig. l(a)| (black curve). 



Fig. 1(a) (red and blue curves) shows that at steady state the power consumption 
of a TCL switches between a constant value and zero in ON and OFF states, 
respectively. The heterogeneity and independence of the TCL dynamics ensure 
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that the time phase of these oscillations are not correlated and do not result in 
noticeable aggregate power oscillations. If we start from a set of arbitrary TCL 
initial states, the heterogeneity of C, R and P values across the population and 
natural temperature fluctuations will cause the aggregate power demand of a large 
ensemble of TCLs settle down to a constant value with fluctuations due to the finite 
size of the population. In this steady state, temperatures of all the TCLs would lie 
within their individual hysteresis dead bands and the temperatures of the TCLs 
attain steady state probability distribution [|2T]| . as presented in Fig. |l(b)[ For 



simplicity, we will use this steady state as our reference state for demonstration of 
response of the population to control signals. To help understand the figures to 
follow, a list of the different marks/signs used is presented in Fig. [2} 

While undisturbed, the TCL dynamics remain uncorrelated, however, some 
forms of control can result in potentially dangerous correlations. For example, 
suppose that our goal is to reduce power consumed by all TCLs during a ten 
minute period and then return all TCLs to their individual dead bands. Naively, 
one may try switching off all TCLs for 10 minutes and then sending a signal 
to all of them to start operating according to the initial conditions [fT7l[T8l [3l[T9l. 
Fig.[3]shows that such a protocol would create a non-steady-state distribution with 
"kinks" and "gaps" in the ON-and OFF-state distributions. 



In Fig. 4(a) we show our simulations of behavior of a population of 10000 



TCL^ in response to such a control signal. At t = 15 hours, all the ON-state 



'We set R, P and C to follow the lognormal distribution in the ensemble of TCLs, which 
was argued to be a reasonable approximation to a real situation 11211 l27l . and we chose them 
to achieve a typical TCL cycle duration of approximately 45 minutes with a typical duty cycle 
approximately 40% at Oamb = 32 "C and 9,^ = 20 °C. Number of TCLs is = 10, 000; width 
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Figure 3: Evolution of the TCL population distribution (y-axis) over temperature (x-axis) when 
aU TCLs are are forced to stay OFF for a certain duration: (a) On receiving signal, all TCLs 
switch to OFF state, (b) The population distribution simply keeps moving to the right, crossing 
the upper deadband hmit, with no natural switching at either of the deadband limits, (c) At the 
end of the desired duration, a signal is sent to TCLs to return to their original state which results, 
in particular, in all TCLs with 6 > 6+ turning ON. (d) After all TCLs return inside the originally 
set temperature range, a kink (higher than normal density of TCLs) and a hole (zero density over 
temperature) circulate with time. Notation is explained in Fig. |2] 
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(a) TCLs switched OFF for 10 minutes 



Time (hrs.) 
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(b) TCLs switched ON for 10 minutes 



Figure 4: (a)(left) Aggregate power consumption when all TCLs are forced to switch OFF for 
10 min at 15 hrs. (a)(right) Red and blue curves show temperature evolution of two individual 
TCLs that are OFF and ON, respectively, at 15 hrs. (b) Same as in (a) but for the case when 
initially all TCLs are requested to switch ON for 10 min. 

TCLs are switched OFF and all the TCLs are kept in switched OFF state for 
10 min. The aggregate TCL power, in Fig. 4(a)P eft), drops to zero, however, after 
receiving the signal to return to the initial state, the aggregate power does not 
return to the initial steady state value (60MW). Instead, the population consumes 
much more power (with a peak above lOOMW). During the ten minutes without 
the temperature control, many of the controlled temperatures increased beyond the 



customer upper dead band 6'_|_, as shown in Fig. 4(a) 'right), and a sudden signal to 
return to the initial temperature range forces a larger fraction of TCLs to switch 



of the deadband A = 1 °C; Parameters C, R and P have mean values 3 kW/°C\ 2 "C/kW and 
14 kW, respectively, and a standard deviation of Cp = 0.07 of the corresponding mean; a zero- 
mean Gaussian noise with standard deviation of 0.052°C/min°'^ was added to ([ij to account for 
natural temperature fluctuations. 
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to the ON state than had originally switched to the OFF state. This phenomenon 
is known as a cold load pickup ||36l |371 ■ Similar synchronization happens when 
TCLs are switched ON for a 10-min interval (Fig. 4(b)[ ). 

After providing beneficial control during the first ten minutes, this naive scheme 
creates an uncontrolled power spike of a similar magnitude and opposite sign. 
Perhaps even worse, the following power consumption relaxes to the steady state 
not monotonously but rather showing pronounced power oscillations, which may 
persist for several cycles depending on the population's heterogeneity. These os- 
cillations, which we will call parasitic oscillations, are sustained by the synchro- 
nization of individual phases of TCLs induced by simultaneous transition of a 
large number of TCLs into the same ON-state. Fig. |4] (left) shows the inverse 
situation when all the TCLs are set to be ON for ten minutes after which the 
originally-ON-TCLs are switched back from OFF to ON resulting in a similar 
damped oscillation. 

Another "naive" approach to control the aggregated power of TCLs is to im- 
pose a sudden shift of dead band positions [|2T1 l35l . as shown in Fig. [5| This 
approach of a sudden shift in deadband limits uniformly across the population 
also creates kinks and gaps in the temperature distribution, which sustain for a 



long duration. Fig. 6(a) shows our numerically calculated aggregated power re- 
sponse of a group of TCLs, which were initially operating in a steady state, after 
their temperature set point was increased by 0.4 "C . As a result, some of the TCLs 



instantly found themselves lying below 9_, as shown in Fig. 6(b) and therefore 
switched OFF resulting in an immediate drop in the aggregate demand. After this 
immediate drop, however, the aggregate demand undergoes large damped oscilla- 
tory mode due to essentially the same reasons as in previous example. 
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Figure 5: Evolution of temperature distribution under sudden shift of deadband, by S, from 
[0°_,e1] to [9-,0+]: (a) As the deadband is shifted, a fraction of the initially OFF TCLs "in- 
stantly" switches to ON, creating a "kink" in the OFF-distribution. (b) While OFF-state TCLs are 
yet to reach the new upper deadband limit, ON-to-OFF switching already occurs at the new lower 
deadband limit, (c) As switching starts at both the new deadband limits, a "hole" is created in the 
ON-distribution. (d) The "kink" and "hole' keep moving around the distribution. 
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15 16 17 15 16 17 14.5 15 15.5 16 16.5 17 17.5 



Time (hrs.) Time (hrs.) Time (hrs.) 

(a) Aggregate response under shift (b) Typical response of individual TCLs 

Figure 6: (a)Aggregate response (left) under temperature setpoint shift (right) by 0.4 °C. (b) 
Individual temperature evolution of two TCLs. 

In summary, the synchronization of TCLs by external control signals may 
induce considerable power oscillations with amplitudes comparable to the total 
power consumed by the population at steady conditions. Control strategies that 
do not avoid such unwanted parasitic power oscillations may be more harmful 
than the power fluctuations that they attempt to remove. 

3. Safe protocol- 1 

We start our discussion of safe strategies to create power pulses in the grid by 
considering a practically less interesting but more easy-to-follow example. Con- 
sider the strategy to control the population, which is illustrated in Fig.|7]and which 
we will term the safe protocol- 1, or simply SP-1. To implement any SP, an im- 
portant assumption is that temperature controllers have additional embedded in- 
structions. Specifically for SP-1, first, we assume that after receiving the control 
signal, TCLs are capable to save the information about the temperature that they 
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are having at this moment. Second, we assume that TCLs are capable to com- 
pare their current temperature with their saved one. Finally, we assume that by 
reaching such a saved temperature, the TCLs can make transitions between ON 
and OFF states purely autonomously, i.e. without waiting for an external signal. 
Such additional instructions are elementary and can be easily added to tempera- 
ture controllers of TCLs in most houses. 




Figure 7: Distribution of TCLs (y-axis) over temperature (x-axis) in the ON and OFF modes 
during stages of safe protocol-1 (SP-1). (a) All TCLs are requested to switch OFF. (b) and (c) 
Intermediate stages of evolution of population distribution (y-axis is not to scale), (d) After all 
TCLs appear again in the OFF mode, TCLs that were in the ON mode before application of 
the protocol make autonomous transition from OFF to ON. As a result, the original steady state 
distribution is reproduced. 

Having such additional capabilities, the generation of a safe pulse can proceed 
as follows: 
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1. By receiving an external signal, TCLs from the ON branch switch to OFF 
(Fig-Ba)). 

2. At this moment, TCLs that switched from ON to OFF save their tempera- 
tures. 

3. All TCLs of the ensemble continue working without changing the position 
of the temperature operation band, as shown in Fig.|7];b)-(c). 

4. When temperature of TCLs that switched OFF at step-1 reaches the same 
temperature as the saved one when it is again in the OFF-mode (this stage 
is achieved after one period of the cycle, as shown in Fig.|7];d)), these TCLs 
autonomously switch to the ON state. 

A similar process works if the power needs to be increased at initial moment, 
by switching TCL's states, initially, from OFF to ON at step-1. 

Fig. [8] shows two simulation results of application of SP-1 to instantaneously 
decrease (Fig. 8(a) I and increase (Fig. 8(b)[ ) power in a heterogeneous population 



of TCLs. While after the immediate change, the aggregate power recovers by 
making a swing in the opposite direction, it quickly settles down back to its origi- 
nal steady state value after one cycle duration of a typical TCL in the population. 
This becomes possible because the original distribution is reproduced after com- 
pletion of stages of SP-1 (Fig. |7];d)). Fig. |8(a)| ^right) illustrates that, while TCLi 



that was OFF at the start of SP-1 at time 15 hrs continues with its natural cycle, 
TCL2 switched its state from ON to OFF and then switched back from OFF to ON 
after a duration equal to its natural cycle period. An equivalent behavior occurs 
when TCLs are initially switched ON, Fig. [8(b) 



Here we note that Fig.|7]shows that the SP-1 ends with equilibrium distribution 
of TCLs over temperature if the population is homogeneous, i.e. if all parameters 
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Time (hrs.) Time (hrs.) Time (hrs.) Time (hrs.) 

(a) TCLs switched OFF (b) TCLs switched ON 

Figure 8: Aggregate power (left) and individual temperature (right) response of TCLs under SP-1. 
Notation is explained in caption for Fig.|4] 

of TCLs are identical with identical cycle duration. A real population would be 
certainly heterogeneous. Nevertheless, Fig. [8] for the power response shows that 
SP-1 does not lead to unwanted power oscillations by the end of the protocol even 
if it is applied to a large heterogeneous population described in the footnote in 
Section II. The reason is that for any TCL in a large heterogeneous population, 
there are many other TCLs with similar parameters. That is, each large popu- 
lation can be considered as a set of many almost homogeneous sub-populations. 
Since a SP does not lead to unwanted power oscillations in any homogeneous 
sub-population, it does not induce such oscillations when it is applied to a full 
heterogeneous ensemble of TCLs either. 

Safe protocol SP-1 has two additional useful features: 

• Power increases almost instantly in response to the external signal so that 
the protocol can be used to offset fast power fluctuations. In reality, of 
course, there is always a time delay between receiving the signal and chang- 
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ing the mode of operation of a TCL, which restricts the speed of control. 
Response time of a realistic TCL to external control, however, is as small as 
a fraction of a minute |fT9l . 

• Since the temperature deadband remains undisturbed, this protocol guaran- 
tees that we never leave the temperature operation band that was set by a 
customer. Hence, application of this protocol should be practically unno- 
ticeable by owners of TCLs. 

• Finally, and this will be a generic property of all safe protocols discussed 
in this article, the safe protocols are also safe from the privacy point of 
view. They do not require the utility company to have information about any 
specific TCL. Only aggregate information about the total power consumed 
by the ensemble is needed to estimate the power response to a SP. 

However, the applicability of SP-1 alone is limited because SP-1 cannot gen- 
erate any extra energy by the time of its completion in comparison to the energy 
generated by the same ensemble at the steady state during the same time. At the 
end of SP-1, the ensemble of TCLs is brought to the initial distribution, exactly 
the same state in which ensemble would be if no external control is applied. Intu- 
itively, the conservation of energy argument suggests that SP-1, while producing 
a nontrivial power pulse, does not save any extra energy by the time of its com- 
pletion, which is indeed the case as we prove in the Appendix. 

4. Safe Protocol-2 

The next protocol, SP-2, shows that the drawback of the previous protocol, 
i.e. the inability to store/release energy by the time of its completion, can be 
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eliminated by choosing a different strategy. In particular, SP-2 creates a power 
pulse with a net release of energy, in comparison to the undisturbed ensemble, 
to the grid. This is achieved during a shift of the positions of TCL's temperature 
operation bands by a fixed amount of temperature. This protocol has been already 
presented previously [I35L so we only describe it here briefly because it will be 
used later in our discussion. 

To describe the SP-2, we should introduce the transition points that are interme- 
diate hysteresis deadband limits that become operative starting from the instant 
the temperature shift is initiated until the new set of deadband limits are applied 
across the population. Let and 6^j^ be the lower and upper deadband limits 
before a shift of the deadband position to the new limits 6'_ = (6'° + 5) and 
= (6'° + (5) . If the temperature setpoint is to be increased by an amount 5, 
then the transition points would be 6'^_ and [6^ + 5) . While for a decrease in set- 
point by an amount 5, the transition points would be (0'^_ — 5) and 6*° . Note that 
positions of the transition temperature points do not coincide with the positions 
of the new dead band limits. 

The SP-2 was constructed as follows: 

1 . A TCL continues to stay in its present state (ON or OFF) until it hits one of 
the transition points. 

2. Once it has reached one of the transition points, the new pair of deadband 
limits, 6^ = (6^° + 6) and 9^ = (^9'^ + 5), starts governing its thermal 
dynamics. 

Here 5 > for the increase in the setpoint and 5 < for the decrease in the 
setpoint. Fig. |9] shows how this protocol works when the setpoint temperature is 
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increased by S. The aggregate power demand of the population of TCLs attains 
a new steady state corresponding to the shifted deadband positions within a ther- 
mostat's cycle duration. Fig. [10] illustrates how SP-2 performs when the setpoint 
is increased.The aggregate response in Fig. |10(aj] shows that the power drops for a 
duration equal to the time period and then stabilizes to the new steady state value 



without any parasitic oscillation. Fig. 10(b) shows two typical responses of indi- 
vidual TCLs. TCL2 was initially OFF and hence simply continues with the new 
deadband limits, while TCLi which was ON at the initiation of SP-2 continues 
with its old deadband limits until it hits the lower deadband limt after which it 
adopts the new limits. 
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Figure 9: Evolution of TCL's distribution over temperature in the ON and OFF modes during 
states of Safe protocol-2 (SP-2), which is based on shifting temperature range hmits from initial, 
iO°_ , 91), to new ones, (9- , 9+). 
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15 16 17 15 16 17 14.5 15 15.5 16 16.5 17 17.5 

Time (hrs.) Time (hrs.) Time (hrs.) 

(a) Aggregate response under SP-2 (b) Typical response of individual TCLs 

Figure 10: Aggregate power and individual temperature response of TCLs under SP-2 when the 
temperature setpoint is increased by 0.4 °C. Notation is explained in caption for Fig.[6j 

Besides the absence of parasitic oscillations, this safe protocol (SP-2) has sev- 
eral new advantages: 

• It can be used to switch the set point temperatures almost in arbitrary range, 
limited only by customer tolerance level. 

• This protocol practically does not require extra switching of the device 
branch per time of the driving protocol. The completion of the protocol 
takes about the cycle duration time and by this time all TCLs switch their 
mode (from OFF to ON or from ON to OFF) only one time. In comparison, 
SP-1 takes about the same time before its completion during which each 
TCL makes 4 mode switching events, while a steady state cycle includes 
only 2 switching events between OFF and ON states per one TCL cycle. This 
is important because more frequent switching of the TCL operation mode 
accelerates the fatigue of the device circuits and may be considered the ma- 
jor cost of application of the control. 
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By careful examination of Fig.|9} one can recognize the reason why SP-2 does 
not lead to unwanted power oscillation: All steps are chosen to avoid creation of 
"kinks" or "holes" in the final distribution of TCLs over temperature. 

The major drawback of SP-2 is its "slowness". Power does not change abruptly 
after receiving a control command. It takes a certain while to reach the extremum 
of power consumption by an ensemble. Hence SP-2, alone, is restricted to very 
specific applications that do not require fast response, e.g. for offsetting relatively 
slow (~30-60 minutes in our case) but possibly large power fluctuations. For ex- 
ample, one can envision a situation when a power line breaks down and one wants 
to decrease the burden by TCLs on the Power Grid during a relatively large time 
interval, e.g. to provide time for a backup coal power plant to increase power 
output. Such situations are commonly resolved by relatively expensive, in com- 
parison to standard power generators, spinning reserves, which annual cost on the 
scale of such a country as the USA is considerable. This justifies the importance 
of the SP-2 for helping to resolve such problems. 

5. Safe Protocols to Generate Short Power Pulses 

TCL's ability to respond to control signals at a fraction of a minute suggests 
that they may not be simply considered as replacement to standard spinning re- 
serves. They can be also used to increase stability of the power supply in the 
power grid by offsetting sharp power spikes. For such applications, it is desirable 
to design control protocols that are safe against synchronization problems but re- 
act much faster and create very short power pulses. While safe protocols SP-1 
and SP-2 could create power profiles without unwanted power oscillations, none 
of them is suitable for offsetting short (2-5 minutes) power spikes because the 
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pulses that they generate continue for a duration of a cycle of a typical TCL in the 
population. 



OFFL 



A 



OFF 
ON 



OFF 




Figure 1 1 : SP-3 based on switching (to OFF) and shifting, followed by SP-2, to bring the popula- 
tion of TCL's to the initial setpoints. 

Here we construct the protocol, referred to as SP-3, that generates a short 
power pulse of possibly large amplitude. If we consider that time of response of a 
power plant to fluctuations is e.g. 8 minutes, then what one would need to increase 
stability of the power supply is to be able to create shorter sharp pulses of duration, 
e.g.. At = 3 minutes with a TCL ensemble and then return all TCLs to initial 
setpoints avoiding sharp pulses of opposite sign and power oscillations. This is 



achieved by the protocol, which stages are shown in Fig. 1 1 The mechanism of 
SP-3, when the need is to decrease power, is as follows: 

1. By receiving the signal, TCLs in the ON mode switch to OFF (Fig.[TTJa)) 
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and save information about their original deadband limits, 6*° and 9^. 

2. All TCLs continue on OFF mode for a duration At (say, 3 min), as shown 
in Fig.[TTJb). It is to be noted, that during this time At, no switching takes 
place at either of the deadband limits. 

3. At the end of this interval, TCLs that were forced to switch the branch at 
step-1 now should switch from OFF to ON mode. The result is shown in 



Fig.fTgc). 

Note that at this stage no "kinks" or "holes" in the distribution were cre- 
ated. Steps 1-3 will produce sharp drop of power consumption for a time 
At and resulting distribution at the end of step-3 will be close to the new 
equilibrium because, after step-3, we did not create any kinks or holes in 
TCL's distribution. These steps, however, were also discussed in Section 2, 
in which we showed that if, following step 3, we simply ask all TCLs to 
return to initial conditions, parasitic power oscillations can be considerable. 
In order to fix this problem, we propose an application of SP-2 to effect a 
slow return of deadband limits. 
4. After step-3, the ensemble is brought to the initial state (Fig. [TTj^d)) by ap- 
plication of a specific version of SP-2 (refer to Section]?]). SP-2 that was de- 
scribed in the previous section requires information about the current dead- 
band limits and the new deadband limits (or, alternatively the amount of 
shift in setpoint temperature), to use as transition points. But the "current" 
deadband temperature limits, at the end of step-3, 9- and 9+ are not exactly 
known for a heterogeneous population. One can show, however, that this 
knowledge is not needed. We can simply instruct all TCLs that switch to 
ON-branch after step-3 to start operation according to the old, saved, tem- 
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perature deadband {9^ to 6*° ) after step-3. As for the TCLs that were in 
the OFF-branch before step- 1 , we instruct them to continue staying on this 
branch for a duration At from the moment they reach the temperature 9^ 
(saved in step-1). This can be achieved by adding timers to the temperature 
controllers so that the timer starts autonomously after reaching 9'^. Only 
after completing such extra At delay in the OFF-mode, these TCLs switch 
from OFF to ON and then set their thermostats to work according to the old 
band position. This set of instructions will not create kinks in the distribu- 
tion and will be equivalent to application of SP-2, with effective transition 
point at 9'^ and 9^, following step-3. Note that step-4 is necessarily slow so 
that sharp and strong drop of power during steps 1-3 is compensated by a 
slow and weak increase of power during step-4. 

Fig. [T2| shows simulation results of the application of SP-3. A negative pulse of 



width 3 min is generated following the steps 1-4 of SP-3, as shown in Fig. 12 (right). 
The sharp and exact positive pulse is followed by a slow and weak change in ag- 
gregate power in the opposite direction for about a cycle duration before returning 
to the original steady state consumption level. A similar result, but in opposite di- 
rection, happens when a positive pulse of width 3 min is generated (Fig. [72] (left)). 



This method is particularly suitable for applications, in which the required 
pulse width is much smaller than the cycle duration, e.g. less than 5 minutes. This 
strategy opens the possibility of an application of TCLs to a class of problems that 
standard spinning reserves cannot resolve. 
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Switched ON Switched OFF 




Time (hrs.) Time {hrs.) 



Figure 12: Power response under SP-3 generating a positive pulse by switching ON (left) or neg- 
ative pulse by switching OFF (right), both of a width of 3 min. 

6. Hybrid protocols 

In previous sections, we described three protocols that generate power pulses 
of different shapes with high degree of control. In real situations, those shapes 
separately may not be sufficient to achieve all utility goals - e.g. creating arbitrary 
shape of power response. The latter goal can be achieved by combining different 
pulses. For example, a train of power spikes can be eliminated by a train of SP-3 
pulses sent to sub-populations of TCLs. However, applying SP-3 alone in a series 
of pulses may not be the best strategy for some goals. For example, if TCLs are 
used to replace standard spinning reserves, then the typical situation that they may 
need to resolve is to provide extra power supply at fixed level for relatively long 
duration, e.g. of 30 minutes. In this section, we show an example of a hybrid 
protocol that can be used in such a specific situation. 

Suppose that in response to a damage of a power line, a spinning reserve 
should react almost instantly and supply extra power in the Power Grid during 
30 minutes. In principle, extra power can be supplied by SP-2 but, alone, it is 
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slow. On the other hand, SP-1 responds "instantly" but does not supply an overall 
extra power. This suggests that if we split our ensemble of TCLs into two parts - 




Figure 13: Combination of SP-1 and SP-2 into a hybrid protocol to generate a (negative) pulse. N 
is the total number of TCLs in the population, p is the fraction of TCLs that receive SP-1 signal 
and (1 — p) is the fraction of TCLs that receive SP-2 signal. 

one that receives SP-1 and one that receives SP-2 then we can generate a desired 
power profile that changes power consumption almost instantly and sustains it at 
the required level for sufficiently long duration. In Fig. [10] (left) and Fig. [8] (right) 
both power profiles of SP-1 and SP-2 take about a TCL cycle duration before 
settling down to a steady state value, and the power profiles experience peaks at 
around the same time in opposite directions. Thus if a section of a population fol- 
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lows SP-1, and the rest of TCLs follow SP-2, the aggregate power can be made to 
follow a square-like pulse profile before settling down after one cycle duration. Of 
course, the challenge is to design the population size undergoing protocols SP-1 



and SP-2, and also the amount of setpoint shift to be used for SP-2. Fig. 13 shows 
a graphical illustration of how a hybrid SP is supposed to work. 

The following parameters are important to quantify and design the protocol: 

• p: the fraction of the population, of size A^, undergoing SP-1 (while the rest 
of the population, of size {N — Np) undergoes SP-2), and 

• 5: the amount of the temperature shift to be used for SP-2. To generate a 
negative pulse, 5 > (i.e. sign{S) = 1) and accordingly in SP-1 all the 
TCLs are to be switched OFF. Opposite actions are to be taken to generate 
positive pulse. 



Fig. 14 presents two sample simulation results of application of this strategy 



for generating a positive (Fig. |14(a)[ ) and a negative pulse (Fig. |14(b)| ). Fig. 1 14(b) 




Time (hrs.) 




Time (hrs.) 



(a) Positive pulse: p = 0.3, S = -0.9 °C (b) Negative pulse: p = 0.36, 5 = 0.9 °C 

Figure 14: Power response under hybrid protocol that combines SP-1 and SP-2. 
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shows that by combining two SPs, we can instantly respond to emergency by 
decreasing power consumed by TCLs and sustain the lower power demand by the 
population almost at constant value during half an hour. During the following half 
an hour the power demand by the population is slowly growing and saturating 
close to the initial value. This is a type of the power profile that is frequently 
needed from spinning reserves ifTTlfTSl . 

7. Conclusion 

We have described and simulated the operation of a class of open-loop safe 
protocols that resolve the TCL synchronization problem nearly eliminating oscil- 
lations of controlled TCL power. These safe protocols have several benefits: 

• absence of synchronization of TCLs and related parasitic oscillations 

• fast control response (limited mostly by communication latency) enabling 
the generation of sharp pulses 

• no requirement for two-way communication, and consequently preservation 
of privacy 

• very limited information about the TCL ensemble is required to estimate 
the response. 

• TCL ensembles can be for both up and down regulation, i.e. TCL power 
can be both increased and decreased 

However, safe protocols also appear to have certain limitations whose impact 
require further investigation: 
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more frequent than normal switchings of TCLs can shorten their Ufetime. 
For example, when SP-3 is used with a small duration 5t, a TCL will un- 
dergo at least two switchings while normally it takes a cycle duration (e.g. 
~60 min in the studied population) to complete two switchings. This effect 
likely limits SP-3 to specific applications to responding to unusually large 
but relatively rare power imbalances such as on the shoulder of wind ramps 

the response of air conditioners to the control signal is usually restricted by 
a delay of about 20 seconds, which restricts duration and precision of SP-3 
application to this class of TCLs 

customer tolerance can limit allowable changes in temperature setpoint re- 
stricting the application of SP-2 

the finite size of a TCL population restricts the total energy that can be com- 
pensated through pulse generation. After obtaining a useful power pulse, it 
takes about the natural TCL cycle time for the switched safe protocols to 
"recover", i.e. to bring switched TCLs back to the customer-defined dead 
band range. Therefore, a given TCL can be used only once per this charac- 
teristic time 

an estimate of at least some aggregated TCL ensemble parameters are still 
desired for accurate control of generated power response. This information 
is likely obtainable from monitoring at the utility level, e.g. by observing 
response from weak short pulses generated by SP-3. An alternative possi- 
bility is to use a limited number of houses with smart meters for two-way 
communication to provide information that is sufficient to estimate basic 
aggregated parameters of the whole population 
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• the safe protocols require a small amount of trivial to realize embedded 
intelligence in each TCL controller. 

If these challenges can be overcome, the potential impact of TCL demand re- 
sponse, and specifically the safe protocols described here, would be large. Current 
estimates show that there are over 130 millions houses in the USA alone with over 
100 millions of them equipped with air conditioners [|38l . Considering a conserva- 
tive estimate of lOOMW reliable power control from air conditioners and a similar 
amount from water heaters in a one million houses, this technology implemented 
on the national scale could provide up to 20 GW power of ancillary services. 
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Appendix A. 

Theorem Appendix A.l. (No-Go Theorem) One cannot generate extra energy 
by switching conditioners between ON and OFF branches along the SP-1 in a 
large heterogeneous ensemble ofTCLs. 

Proof. Consider application of the SP-1 to a large population of TCLs, that are 
initially in the uncorrected steady state. We assume that due to large size of the 
population, it can be partitioned by a number of sub-populations, which still have 
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large sizes, with practically identical parameters of TCLs and ambient conditions. 
It is then sufficient to prove the No-Go Theorem for an arbitrary homogeneous 
sub-population, because if any subpopulation does not produce an excess of en- 
ergy then neither does their ensemble. 

The evolution of temperature of a TCL, in ([!]), is linear both in temperature and 
consumed power. Hence, after averaging ([T]) over the population with random ini- 
tial phases, we find that the averaged consumed from the Power Grid power (P), 
and the averaged temperature {6) of the ensemble also satisfy a linear relation: 

«|w = -(ni)) + /3(W-7), (A.l) 

where a, (3, and 7 are constants that characterize the ensemble. They can be 
expressed via the number of TLCs A^, the efficiency of the TCLs, and parameters 
R, C and Oamb- At the steady state, we would have ^ {9) = 0, so that the average 
power consumed by the ensemble in the steady state is (Pst) = (^{{Ost) — 7), 
where (^^st) is the average temperature in the unperturbed ensemble of TCLs. The 
total energy consumed by the ensemble in the steady state during some time r 
then would be 

Wst{T) = P{{e)--f)r. (A.2) 

Consider now the total energy, Wtot, generated by the TCL ensemble during 
the time tp between the initial and the final stages of the SP-1: 

W^tot = r dt{P{t)) = [3 r dtm - 7), (A.3) 

Jo JO 



where we used (A.l ) and the fact from Section III that, for SP-1, at time tp, the 
final distribution of TCLs over temperature and ON/OFF states coincides with 
the initial distribution. This means that the average temperature of the ensem- 
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ble coincides with the initial one, so that integrating the Ihs of ( |A.1[ ) we obtain 

Here we note that the time tp coincides with the period of the cycle of a single 
TCL. Indeed, if we trace the evolution of any TCL of the ensemble during the SP- 
1 stages, we find that it goes purely autonomously during the period tp through 
all stages of the considered temperature operation band, just like TCLs in nonper- 
turbed ensemble except switching the branches at the beginning and the end of 
the protocol. We also note that at the steady state, the averaged over time power 
consumed by one TCL during the cycle of its operation coincides with the power 
averaged over the large population of TCLs with the random phase distribution. 
This means that, for a single TCL in a homogeneous population, jj'' 9dt = {9st)tp- 



Hence, (A. 3 1 can be rewritten as 



tot 



/3((^st)-7K 



(A.4) 



But (A.4) is exactly the energy Wst in (A. 2) consumed by the TCL ensemble 
during time tp without application of the external control. Hence the excess energy 

is W^tot - W^st = 0. □ 
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